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PREFACE 


There  Is  a  growing  literature  and  interest  in  methods  for  quantifying  subjective  judg¬ 
ments.  Several  ongoing  Air  Force  efforts  utilizing  subjective  judgment  have  come  to  the 
authors’  attention.  Mission  Area  Analysis  requires  subjective  estimates  of  a  large  number  of 
parameters.  Long-range  planning  repeatedly  draws  on  judgments  about  the  fixture  importance 
and  worth  of  plans  and  geographical  areas.  The  Constant  Quest  project,  directed  by  the 
Reediness/NATO  Coordination  Board,  highlighted  the  importance  of  subjective  judgments  in 
evaluating  command  and  control  systems. 

Thomas  Saaty  of  the  University  of  Pennsylvania  has  advanced  e  popular  tod  for  quan¬ 
tifying  and  scaling  the  worth  of  a  set  of  objects  or  entities.  For  problems  tint  fit  the  Saaty 
framework,  this  report  details  an  improvement  on  Saety’s  "eigenvector”  technique  that  is 
easier  to  use  and  more  amenable  to  statistical  inferences. 

This  report  was  prepared  under  the  Project  AIR  FORCE  research  study  effort,  "Evolving 
Concepts  for  Long-Range  Planning."  It  wee  originally  published  in  1900  and  has  been 
expanded,  clarified,  and  updated  in  1966,  as  part  of  the  Project  AIR  FORCE  Resource  Manage¬ 
ment  Program’s  concept  formulation  and  exploratory  rsssarch  activity. 


SUMMARY 


Let  {£i,£s, . . . ,  £»}  be  a  collection  of  obj<  '=?  or  entities  that  are  in  eome  eenee  compa¬ 
rable.  The  At  may  be  alternative  plans  to  achieve  some  goal,  alternative  objects  that  have 
same  eoaspareble  utfity,  or  generally  a  collection  of  entities  that  have  varying  degrees  of  some 
value.  A  vector  uv,ut, ...,«»  is  called  a  ratio  scale  for  the  collection  if,  fbr  each  * 
and  j,  Uf/Uf  is  the  ratio  of  the  value  of  Et  to  the  value  of  Ej. 

An  ingMctant  application  of  ratio  scales  is  in  the  study  of  hierarchies.  Suppose  that  fbr 
each  level  of  a  hierarchy  there  is  a  ratio  scale  fbr  the  value  of  objects  at  that  level  relative  to 
any  object  at  the  next  level  ap.  The  ratio  ecalea  for  various  levels  can  be  combined  muhiplica- 
tively  to  give  a  view  of  the  entire  hierarchy.  Because  hierarchies  are  used  to  model  complex 
systems  in  many  important  military  and  industrial  applications,  the  estimation  of  the  ratio 
scales  deserves  considerable  attention. 

Suppose  that  a  ratio  scale  t*i, «*, . ...  u,,,  14  >  0,  fbr  objects  EuEt, . . . ,  E„  exist a  but  is 
not  known.  Let  <ty,  i,j  -  1,2; ...  ,n  be  subjective  estimates  of  Uj/u/  made  by  a  judge.  In  par¬ 
ticular,  we  assume  a*  -  1  fbr  each  *,  and  -  l/ty.  The  matrix  A  -  [<ty]  of  subjective  pair¬ 
wise  comparisons  is  called  a  judgment  matrix.1 

If  the  judge  is  perfectly  consistent  in  making  estimates,  the  matrix  A  will  satisfy  the  con¬ 
sistency  criterion 


OiiOfk  -  a*  tor  each  i,j,k  . 

If  this  condition  is  met,  any  column  of  the  matrix  A  gives  a  ratio  scale  for  {Ex,Ei, 

However,  judgments  are  frequently  inconsistent,  and  judgment  matrices  rarely  satisfy  the  con¬ 
sistency  criterion.  A  mathematical  procedure  is  required  fbr  estimating  an  underlying  ratio 
scale  baaed  on  an  inconsistent  judgment  matrix  A. 

Thomas  Saaty  (1977a-d)  argues  that  the  “dominant*  right  eigenvector  corresponding  to 
the  maximal  eigenvalue  should  be  used  to  estimate  the  underlying  scale.  The  argument  vk 
The  dominant  eigenvector  is  a  continuous  ftmctkm  of  the  dements  of  the  matrix,  and,  if  the 
matrix  is  consistent,  the  eigenvector  gives  the  unique  (to  within  scalar  multiplication)  scale. 
Thus,  if  the  elements  of  the  matrix  get  perturbed  slightly  in  the  process  of  bring  subjectively 
quantified  by  a  judge,  the  dominant  eigenvector  will  return  a  scale  only  slightly  different  from 
toe  scale  of  an  underlying  consistent  judgment  matrix. 

Although  the  classical  analyst  may  worry  about  uniform  continuity  or  other  erudite  intri¬ 
cacies  of  this  argument,  we  are  worried  about  a  more  basic  oversight:  The  eigenvector  is  not 
the  only  continuous  vector-valued  function  of  judgment  matrices  that  yields  the  correct  scale 
when  the  matrix  happens  to  be  consistent  There  are  many  others,  including  the  vector  of  row 
sums,  the  vector  of  the  inverse  of  column  sums,  any  column  of  the  matrix,  and  the  whole  ring 
generated  by  positive  linear  combinations  of  these  and  other  solutions. 

We  are  aware  of  the  desirable  properties  of  the  eigenvector  in  characterizing  a  linear 
operator  and  its  spectral  decomposition,  but  the  immediate  relevance  of  these  properties  to  this 
estimation  problem  seems  open  to  question.  In  most  estimation  problems,  the  wealth  of  statis¬ 
tical  literature  on  estimation  procedures  and  their  properties  has  enhanced  understanding  of 


‘la  practice,  e  judge  is  uauslljr  salad  to  avgipiy  the  n(n  -  1V2  caper 
SHtris  ia  defined  by  the  above  relatioaaUp. 
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thi  problem.  Below  we  relate  this  particular  problem  to  well-known  statistical  The 

fsoowtric  men  vector*  v  -  vuvs, ...  ,t>m,  given  by 
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•bid*  satisfies  the  continuity  and  consistency  criteria  Saaty  uaee  to  defend  the  dominant 
eigenvector;  has  several  other  desirable  traits:  In  certain  circumstances,  ft  ia  statistically 
optimal  and  gives  rise  to  an  estimate  of  scales  and  a  measure  of  consistency  that  have  known 
statistical  cBetributkms.  In  empirical  studied  reported  hem  it  seems  to  do  as  well  a*  or  better 
than,  the  eigenvector In  preserving  rank  order.  In  adcfithm,  it  la  supported  by  a  literature 
describing'  methods  of  handling  a  wealth  of  variations  of  the  pwthm,  inrimfing  missing  ***** 
and  multiple  judges. 
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I.  INTRODUCTION 


Over  the  past  three  decades,  psychometric,  military,  and  industrial  researchers  have 
directed  considerable  effort  to  the  quantitative  analysis  of  subjective  data.  Analytic  tools 
building  upon  the  subjective  judgments  of  experts  have  been  used  in  such  diverse  fields  as 
energy  policy  analysis,  marketing  research,  economic  forecasting,  and  military  planning.  Prob¬ 
lems  amenable  to  the  analysis  of  subjective  information  abound,  and  numerous  methods  have 
been  proposed  for  acquiring  and  treating  judgmental  data. 

One  question  that  arises  in  the  treatment  of  subjective  data  is  how  to  construct  a  scale  of 
relative  merit  for  a  collection  of  objects  or  activities  based  upon  subjective  comparisons  of  each 
pair  in  the  collection.  For  example,  consider  a  collection  of  three  objects,  labeled  A,  B,  and  C. 
Suppose  that  an  expert  believes  A  has  twice  the  merit  of  B,  B  has  three  times  the  merit  of  C, 
and  A  has  six  times  the  merit  of  C.  It  is  natural  to  construct  a  scale  of  relative  merit  for  A,  B, 
and  C  as  (1,  1/2,  1/6).  However,  suppose  that  the  expert  says  A  has  twice  the  merit  of  B  and 
B  has  three  times  the  merit  of  C,  but  A  has  only  four  times  the  merit  of  C.  In  this  case,  it  is 
not  so  easy  to  decide  upon  a  scale  for  A,  B,  and  C.  This  sort  of  inconsistency  is  common  in 
human  judgments,  especially  when  complicated  issues  are  involved. 

Thomas  Saaty  of  the  University  of  Pennsylvania  has  developed  a  matrix  eigenvector  pro¬ 
cedure  for  constructing  scales  of  merit  based  on  inconsistent  pairwise  comparisons.  The 
method  has  been  applied  in  a  wide  variety  of  planning  and  decision  problems. 

This  report  presents  an  alternative  approach  that  is  preferable  to  the  eigenvector  pro¬ 
cedure  in  several  important  respects.  The  proposed  procedure  is  derived  within  a  statistical 
framework  and  is  compared  with  the  eigenvector  method  on  the  basis  of  theoretical  and  empir¬ 
ical  considerations. 

The  remainder  of  this  introduction  discusses  the  motivation  for  dealing  formally  and 
quantitatively  with  subjective  information  and  provides  a  brief  review  of  some  of  the  literature. 
Section  II  provides  a  short,  nonrigorous  discussion  of  the  eigenvector  method  and  the  proposed 
method  for  utilizing  subjective  judgments  in  quantitative  analysis.  An  example,  illustrating  the 
use  of  the  two  methods  as  well  as  similarities  and  differences  in  their  results,  is  introduced  in 
this  section  and  examined  throughout  the  report. 

In  Section  III  we  give  rigorous  definitions  and  develop  a  framework  for  treating  the  esti¬ 
mation  problem  with  classical  statistical  techniques.  Section  IV  provides  a  mathematical  treat¬ 
ment  of  the  eigenvector  method.  Section  V  deals  with  the  application  of  subjective  judgment 
methods  to  the  study  of  hierarchical  structures.  The  example  introduced  in  Section  II  is  con¬ 
sidered  in  further  detail  there. 

Section  VI  introduces  the  geometric  mean  vector  and  gives  theoretical  justification  for  its 
use  as  an  estimator  of  subjective  scales.  In  Section  VII  we  define  a  statistical  measure  of  con¬ 
sistency  for  subjective  judgment  matrices.  Section  VIII  presents  results  of  a  Monte  Carlo 
study  comparing  the  two  methods.  Section  IX  considers  in  greater  detail  the  example  intro¬ 
duced  in  Section  II  and  expanded  on  in  Section  V. 


QUANTITATIVE  ANALYSIS  OF  SUBJECTIVE  DATA 

The  quantification  of  subjective  data  is  essential  for  dealing  with  a  wide  class  of  problems 
whose  solution  by  other  methods  would  be  extremely  difficult  or  impossible.  Such  problems 


are  often  amorphous  and  vaguely  stated.  They  involve  large,  multifaceted  issues  of  importance 
to  decisionmakers  and  interest  groups  with  diverse  backgrounds  and  biases.  Their  outcomes 
may  determine  the  allocation  of  large  sums  of  public  money  and  impinge  critically  on  the  pub¬ 
lic  internet.  Moreover,  some  facets  of  the  problems  may  lack  any  well-defined,  scalar-valued 
measures  of  merit.  Even  if  there  are  appropriate  measures,  the  collection  of  relevant  objective 
data  might  be  prohibitively  expensive  or  impossible. 

Such  problems  frequently  arise  in  the  assessment  of  future  needs  for  large  organisations. 
As  an  example,  consider  the  problem  of  long-range  planning  in  the  U.S.  Air  Force.  This  prob¬ 
lem  involves  a  great  many  interrelated  issues:  the  effects  of  political  and  economic  factors  on 
national  security,  the  importance  of  various  geographic  regions  of  the  world  to  U.S.  interests, 
the  threat  posed  by  conflicts  of  different  types  in  different  regions,  the  current  strength  of 
forces  to  deal  with  such  conflicts,  and  so  on.  Although  it  might  be  possible  to  define  objective 
yardsticks  to  deal  with  some  of  these  issues,  it  certainly  is  not  possible  for  all  of  them.  For 
some  issues,  subjective  judgments  of  relative  importance  or  value  are  the  only  measures  avail¬ 
able. 

In  some  problems  the  best  information  available  is  subjective,  so  why  ia  quantitative 
analysis  desirable  at  all?  Why  not  just  ask  the  experts  to  make  plans  and  derisions  baaed  on 
an  informal,  intuitive  analysis?  In  fact,  problems  that  are  not  amenable  to  hard  analysis  are 
frequently  resolved  through  intuitive  analysis  by  experts  and  decisionmakers.  However,  there 
are  several  good  reasons  for  using  a  formal,  quantitative  approach  in  these  problems. 

A  formal  analytic  framework  gives  structure  and  definition  to  an  amorphous  mass  of  data. 
It  allows  the  decisionmaker  to  consider  relevant  information  systematically  and  to  examine 
options  and  consequences  one  at  a  time.  In  such  a  framework,  the  analyst  can  break  an 
unmanageable  problem  into  manageable  parts  and  then  synthesise  information  about  the  parts 
in  a  rational  fashion. 

A  formal  analytic  framework  also  permits  sensitivity  analysis  on  alternative  judgments. 
When  a  problem  is  considered  within  a  formal  framework,  tradeoffs  among  alternative  judg¬ 
ments  can  be  spelled  out  explicitly,  and  the  effects  of  variations  in  subjective  judgments  on 
outcomes  can  be  studied.  Sensitivity  analysis  may  even  provide  a  basis  for  resolving  different 
points  of  view. 

Perhaps  the  greatest  advantage  of  a  formal  analysis,  especially  in  governmental  policy¬ 
making,  is  that  it  is  repeatable.  Formal  analysis  provides  the  audit  trial  that  ia  so  important  in 
matters  involving  extensive  allocation  of  public  resources  and  impinging  on  the  public  interest 

Research  literature  on  the  use  of  subjective  information  emphasises  three  major  issues: 
how  to  elicit  meaningful  subjective  judgments  from  individuals  or  groups,  how  to  synthesise 
subjective  and  objective  data  obtained  from  various  facets  of  a  large  problem,  and  how  to  con¬ 
struct  measurement  scales  based  on  subjective  information.  Following  is  a  review  of  some  of 
the  literature  related  to  each  of  these  issues. 


ELICITING  SUBJECTIVE  JUDGMENTS 

Methods  for  eliciting  subjective  judgments  have  received  considerable  attention  in  opera¬ 
tions  research  and  forecasting  literature.  Two  such  methods  are  war  gaming  and  scenario  writ¬ 
ing,  both  of  which  are  used  extensively  in  military  planning  to  provide  insights  into  possible 
future  environments  and  needs. 

Much  of  the  literature  on  eliciting  judgments  deals  with  the  problem  of  acquiring  a  collec¬ 
tive  expert  opinion  free  from  the  usual  negative  effects  of  group  pressure.  An  important 


method  in  this  category  is  the  Delphi  technique,  a  controlled  feedback  procedure  originally 
developed  by  reeearchere  at  The  Rand  Corporation  (Gordon  and  Hateeor,  1864).  In  Delphi,  a 
researcher  interrogates  a  group  of  everts  individually  concerning  their  opinion#  on  pn— ihle 
future  events.  The  reeearcher  assembles  means  and  quaitilaa  for  quantitative  data  thus 
obtained  and  presents  them  individually  to  group  memheci  along  with  arguments  and  oom- 
ments  made  by  other  members.  Group  members  can  then  revise  their  judgment!  The  pro¬ 
cedure  ie  repeated  until  the  range  of  judgment!  narrows.  Tba  controlled  feedback  marhaniam 
in  Delphi  makes  it  possible  for  a  group  of  experts  to  avoid  the  usual  social  pressures  of  open 
discussion.  The  method  has  been  used  in  many  military  and  industrial  applications  (see,  #4., 
Ayres,  1969;  Linstone  ami  Turoff,  1975). 

The  Delphi  technique  has  given  rise  to  several  modifications.  The  Probe  method 
designed  by  researchers  at  TRW  for  forecasting  technological  events  combines  Delphi  with  o 
timing  chart  structure  so  that  events  can  be  considered  in  sequence  (North  and  Pyles,  1968). 
The  method  of  qualitative  controlled  feedback  proposed  by  Press  (1979)  is  similar  to  Delphi  in 
that  it  uses  a  controlled  feedback  loop  to  aid  groups  in  arriving  at  judgments,  but  it  differs  in 
that  at  each  iteration,  members  are  supplied  only  arguments  and  comments  from  the  group, 
with  no  information  about  the  quantitative  distribution  of  group  answers. 

SYNTHESIZING  DATA  IN  LARGE  PROBLEMS 

Economists  and  statisticians  have  proposed  various  methods  for  breaking  large  problems 
into  smaller  pieces  and  quantitatively  synthesising  subjective  and  objective  data  from  the 
pieces.  One  of  the  moat  popular  of  them  methods  is  multi-attribute  utility  tinny,  which  pro¬ 
vides  a  framework  for  selecting  an  optimal  decision  from  among  multiple  alternatives  when 
some  effects  of  the  decision  can  be  measured  only  subjectively.  The  expected  value  of  each 
alternative  is  determined  as  a  function  of  the  decisionmaker's  preferences  for  the  possible 
consequences  and  the  probabilities  that  the  alternative  will  lead  to  thorn  consequences  The 
probabilities  are  generally  determined  from  subjective  judgments.  Soma  of  the  decisionmaker’s 
preferences  are  determined  on  the  basis  of  subjective  indexes  such  as  aesthetic  appeal,  and  oth¬ 
ers  are  determined  on  the  basis  of  objective  measures  such  as  coat  The  alternative  with  max¬ 
imum  expected  value  is  chosen  as  the  optimum  decision. 

The  mathematical  foundation  for  multi-attribute  utility  theory  was  laid  by  von  Neumann 
and  Morgenstem  (1947).  Application  of  the  theory  to  business  problems  was  pioneered  by 
Raiffo  and  extended  by  Keeney  and  others.  The  theory  has  been  applied  to  many  problems  in 
industrial,  government  and  military  settings  (see  Keeney,  1973;  Keeney  end  Nair,  1974;  Kel¬ 
ley,  1976;  MacCrimmon,  1969;  Raiffe,  1969).  A  book  by  Keeney  and  Raiffa  (1976)  gives  an 
excellent  treatment  of  the  subject 

A  similar  method  was  applied  to  military  problems  in  a  1968  Master’s  thesis  by  Wells, 
who  gives  a  detailed  framework  for  wssesaing  the  relative  desirability  of  existing  or  proposed 
weapon  systems.  System  desirability  is  determined  as  a  function  of  feasibility,  cost,  and  an 
attribute  Wells  calls  “military  worth.”  Wherever  possible,  objective  meamrse  are  used  to 
evaluate  these  three  factors,  mid  expert  judgment*  us  used  where  time  are  no  objective  mea¬ 
sures.  In  particular,  military  worth  is  an  aggregate  property  evaluated  by  analysing  a  complex 
hierarchy  and  subjective  scales  for  several  variables.  The  Honeywell  Corporation  used  Wells’s 
method  in  a  military  planning  modal  called  PATTERN  (Sigford  and  Parvin,  1966).  A  detailed 
description  of  the  method  can  be  found  in  Wells  (1967). 
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be  yimd  in  terms  of  hierarchies  of 
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Section  V. 


CONSTRUCTING  MEASUREMENT  SCALES 

Many  methods  have  been  developed  for  constructing  scales  of  msesurement  based  on  sub¬ 
jective  data.  Several  books  and  hundreds  of  articles  have  been  written  about  these  methods.  A 
classic  reference  for  early  contributions,  especially  for  work  on  psychophysical  scabs,  b 
Torgerson’s  (1068)  book. 

Churchman  and  Ackoff  (1064)  dkl  pioneering  work  in  the  area  of  estimating  scalas  of 
values  for  decision  problems.  They  used  a  criterion  of  additive  order  consistency  to  estimate 
miw  from  iuccmiivu  ■uojsciivs  juafiMiiti.  iwir  pcpM  CMcnDM  nvmi  ippuciu(n»  to 
Industrial  problems.  Wells  and  others  later  applied  the  Phtirchtnen  and  Ackoff  method  in  mil 
itary  decision  problems. 

Much  of  the  literature  on  subjective  scales  concerns  the  estimation  of  scales  from  pair¬ 
wise  comparison  data.  Much  statistical  work  in  tide  ana  goes  under  the  name  “paired  com- 
pmom  in  ini  ttiupim  pttfpa  oonpanion  oiponniroty  mcd  oi  nv*m  judge*  ronnnw  a 
number  of  objects  two  at  a  time  and  states  which  of  the  two  objects  b  preferred.  No  indication 

nM  B  Mai  n*4a  1>W AAA  Malaga  4lkaa>  iaaaJ  ta*  A  aAa. 

vi  soengtn  m  psewrenoe  is  given,  trues  rrom  uwss  paueu  compeneons  are  wan  swm  m  a  am- 
tbtiffi)  model  to  —**!—*»  a  ***'*  of  pftfwfw*  foe  the  Soldi  an  expertateut  — tgto  ba 

used  by  marketing  researchers  to  determine  the  relative  taste  appeab  of  several  new  food 


A  good  reference  for  the  statistical  theory  of  paired  comparisons  b  David's  (1968)  book. 
A  bibliography  of  meant  articles  on  the  subject  was  compiled  by  Davidson  and  Farquhar 
(1978). 

Saaty  has  proposed  another  method  for  estimating  subjective  scabs  using  pairwise  com¬ 
parisons  in  which  a  singb  judge  makes  pairwise  comparisons  of  a  number  of  objects.  For  each 
pair,  the  judge  states  not  only  which  object  b  preferred,  but  to  what  degree  that  object  is  pre¬ 
ferred  over  the  other.  A  preference  scab  b  determined  for  the  objects  baaed  on  an  eigenvector 
enalyeb  of  the  matrix  of  pairwise  comparisons. 

8aaty  has  published  many  articles  (tee  the  bibliography)  describing  the  eigenvector  pro¬ 
cedure  for  estimating  subjective  scales  and  illustrating  the  usefulness  of  this  procedure  in 
analysing  complex  hierarchical  structures.  He  has  applied  the  procedure  in  a  broad  range  of 
problems  in  the  social  sciences  (Seaty,  1977b;  Saaty  and  Bennett,  1977;  Alexander  and  Saaty, 
1977).  The  procedure  has  also  gained  acceptance  in  military  applications  and  b  currently 
being  used  as  a  tool  in  Air  Foroa  long-range  planning. 

If  one  aaaumaa  that  an  analysis  problem  b  in  the  context  that  Saaty  has  been  writing 
about,  then  the  results  of  thb  report  imply  that  the  geometric  mean  b  preferable  to  the  eigen¬ 
vector  solution.  However,  Veit  and  Ceilero  (1981)  and  Veit,  Calfero,  and  Rose  (1962,  1984) 
bettave  that  many  problems  may  not  be  multiplicative  at  all  and  may  not  fit  the  Saaty  context 


n.  PAIRWISE  COMPARISONS,  THE  JUDGMENT  MATRIX,  AND 
THE  ESTIMATION  PROBLEM 


Consider  the  problem  of  purchasing  a  new  car.1  Suppose  that  a  preliminary  investigation 
yields  five  specific  makes  that  seem  appropriate.  The  price  of  each  make  is  known,  and 
although  some  other  measures  of  merit  may  have  been  quantified  (principally  performance 
measures),  the  important  subjective  question  of  how  much  each  car  satisfies  tire  overall  needs 
is  difficult  to  quantify. 

We  will  attempt  to  assign  to  each  make  of  automobile  an  estimate  of  utility  in  such  a  way 
that  if  vn  is  the  utility  of  the  rth  make,  then  Uj/u/  is  a  measure  of  tire  preference  of  the  rth 
make  to  the/th  make.  The  vector  u\,ut, . . . ,  a*  will  be  called  a  ratio  scale. 

Some  aspects  of  the  usefulness  of  such  a  ratio  scale  are  immediately  apparent.  We  could, 
in  this  example,  chooee  between  the  cars  on  the  bans  of  utility  per  dollar  of  initial  cost  or,  with 
more  foresight,  on  the  basis  of  utility  per  dollar  of  expected  life  cycle  cost. 

To  estimate  the  vector  of  utilitiee  ui,u*, . . . ,  Ug  Saaty  has  suggested  the  following  pro¬ 
cedure  (see  especially  1977b):  We  construct  a  matrix  composed  of  our  subjective  estimates  of 
the  ratios  of  the  utilities  of  all  possible  pairwise  combinations,  so  that  the  element*  of  the 
matrix  A  are  our  estimates  of  iq/ity.  Thus  we  know  that  the  diagonal  elements  are  given  by 
a#  -  1,  i  -  1, . ...  6.  Additionally,  the  lower  off-diagonal  elements  are  determined  by  the 
upper  off-diagonal  elements:  a#  -  1/a#. 

Saaty  (1977b)  proves  that  in  this  case  the  matrix  A  has  a  maximal  eigenvalue  and  a 
corresponding  eigenvector  (the  dominant  eigenvector)  all  of  whose  component*  are  positive. 
Saaty  proposes,  primarily  with  empirical  justification,  that  this  dominant  eigenvector  be  used 
as  an  estimate  of  the  ratio  scale. 

Suppose  that  when  we  form  our  estimate*,  the  relative  utility  of  Make  1  to  Make  2  is  2, 
of  Make  1  to  Make  3  is  1/9,  at  Make  1  to  Make  4  is  1/8  and  of  Make  1  to  Make  5  is  1/7. 
Then  the  first  row  of  our  judgment  matrix  has  the  form 

1,  2,  1/9,  1/8,  1/7  . 

Continuing,  suppose  that  w*  have  filled  in  the  upper  off-diagonal  of  our  judgment  matrix: 

12  1/9  1/8  1/7 

1  1/9  1/8  1/7 

16  4 

1  1/7 

1 

Then,  in  view  of  reciprocal  symmetry  we  have 


1 

2 

1/9 

1/B 

1/7 

1/2 

1 

1/9 

1/8 

1/7 

A-  9 

9 

1 

6 

4 

6 

6 

1/8 

1 

1/7 

7 

7 

1/4 

7 

1 
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Continuing  with  this  example  we  couymte  the  dominant  eigenvector  w  of  this  matrix  and 

cat: 

.0378 
.0294 
to  -  .8239 
.1131 
.2968 

Thus,  in  this  case,  our  estimate  of  the  utility  of  the  first  make  is  .0378  and  of  the  third 
make  is  .5239.  For  a  detailed  treatment  of  this  procedure  see  Saaty,  1977b;  Saaty  and  Bennett, 
1977. 

This  example  m  discussed  in  more  detail  in  Sections  V  and  IX,  where  it  is  expanded  to 
illustrate  the  value  of  ratio  scales  in  analysing  hierarchical  structures.  In  application  to  hierar¬ 
chies  ft  is  aammed  that  the  objects  at  each  level  of  the  hierarchy  depend  on  the  otgects  of  the 
next  lower  level  in  some  way.  The  procedure  enables  the  near  to  estimate  the  influence  each 
object  in  a  level  has  on  all  the  objects  or  goals  in  superior  levels. 

For  problems  where  the  eigenvector  procedure  is  useful  there  is  another  estimation  pro¬ 
cedure  that  is  preferable  in  several  reepecte. 

Where  Saaty  would  estimate  the  utility  of  the  tth  object  with  the  ith  component  of 
the  dominant  eigenvector,  we  argue  that  a  better  estimator  is  given  by  the  vector 

v  -  t>i,t>2, ...  ,vn,  where 

7-i 

is  the  geometric  mean  of  the  elements  in  the  fill  tow  of  A.  In  the  example  above  tins  yields 
the  following  estimates: 


Object 

Eigenvector 

Geometric 
Mean  Vector 

1 

.0378 

.0409 

2 

.0294 

.0310 

3 

.5239 

.6307 

4 

.1131 

.1132 

5 

.2958 

.2842 

Compared  with  the  dominant  eigenvector,  the  geometric  mean  vector 

1.  Is  statistically  better, 

2.  Is  easier  to  calculate; 

3.  Gives  rise  to  a  more  meaningful  measure  of  consistency  that  has  known  statistical 
properties,  allowing  tests  of  hypotheses,  confidence  interval  estimation,  etc.; 

4.  Gives  rise  to  estimates  of  utility  with  known  statistical  properties,  allowing  tests  of 
hypotheses,  confidence  interval  estimation,  etc.; 

5.  Is  supported  by  statistical  literature  discribing  methods  of  handling  a  wealth  of  varia¬ 
tions  of  the  problem; 

6.  Is  rooted  in  a  mathematical  approach  to  estimation  that  provides  an  intuitive  under¬ 
standing  of  the  problem  and  a  mens  for  ssrrrrinf  suitability  of  the  method. 


.  v 
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HI.  CONSISTENT  MATRICES  AND  RATIO  SCALES 


Consider  a  set  of  n  activities  or  objects  Ei,Ea,  ...,E„,  which  contribute  to  some  objec¬ 
tive.  Suppose  the  activities  can  be  ranked  on  a  ratio  scale  ....  u,),  u,  >  0,  so  that 

Uj/Uf  measures  the  degree  to  which  £,  is  mors  important  than  Ej  in  achieving  the  objective. 
In  particular,  m/Uj  >  1  if  Ei  m  mote  important  than  Ej.  Let  A  -  [<ty]  be  the  nxn  matrix  of 
pairwise  comparisons  of  EuEt,  . . . , given  by 

i,j  -  1,2, ....  n  .  (3.1) 


i,j  -  1,2, ....  n  ,  (3.2) 


i  -  1,2, ....  n  . 

A  square  matrix  A  with  positive  entries  satisfying  (3.2)  will  be  called  a  judgment  matrix. 

It  follows  immediately  from  (3.1)  that 

<Hiajk  -  a*  •  (3.3) 

A  matrix  with  positive  entries  satisfying  (33)  is  said  to  be  consistent.  It  is  easy  to  see  that 
every  consistent  matrix  is  a  judgment  matrix. 

Let  A  be  an  arbitrary  consistent  matrix.  Because 

a>*  -  for  any;*  , 

aU 

every  element  of  A  can  be  determined  from  the  first  row  of  A.  It  follows  that  A  is  a  matrix  of 
rank  one  with  exactly  one  nonzero  eigenvalue.  Moreover,  it  follows  from  (3.3)  that 

A*  -  nA  . 

Thus  any  column  of  A  is  an  eigenvector  of  A,  and  the  single  nonzero  eigenvalue  of  A  is  n. 

Let  w  -  wi,u;s, . . . ,  wn  be  any  eigenvector  corresponding  to  the  eigenvalue  n.  For  any  k, 
the  *th  column  of  A  is  an  eigenvector  corresponding  to  the  same  eigenvalue;  therefore  for  each 
i  and;, 


Om  -  - 

“f 


Then  A  has  the  property  that 


and  in  particular 


on  -  1. 


for  some  c  *  0,  and  therefore 


wi-catk 
Wj  -  coy* 


m 

Wj 


0 a  • 
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Thu*,  w  ie  •  ratio  Mile  for  A.  In  fact,  it  i»  dear  that  there  are  infinitely 
each  one  corresponding  to  a  different  scalar  multiple  of  the  hth  column  of  A. 
The  normalised  eigenvector  with  components 

Wi 


such  amine. 


2>i 

*-i 


tM  the  particular  scale  that  Saaty  (1977b)  associates  with  the  consistent  matrix  A. 

A  ratio  scale  corresponding  to  a  consistent  matrix  A  can  be  derived  in  several  ways.  Any 
column  of  A  is  such  a  seals.  The  vector  of  reciprocals  of  elements  of  an  arbitrary  row  of  A  is 
also  a  ratio  scale  for  A.  It  is  easy  to  see  that  the  vector  r  of  row  sums  defined  by 


J-i 


and  the  geometric  mean  vector  v  defined  by 


*  -  IK" 


also  provide  ratio  scales  for  A.  When  these  scales  are  normalized,  they  are  equal  to  the  nor¬ 
malized  eigenvector  scale  for  a  consistent  matrix. 


IV.  SAATY’S  NORMALIZED  EIGENVECTOR  SCALE 


Consider  again  the  activities  E\,Et, . . .  ,En  that  contribute  to  some  objective.  Suppose  a 
judge  makes  pairwise  comparisons  on  some  scale  of  the  relative  importance  of  each  pair  of 
activities  with  respect  to  the  underlying  objective.  If  represents  the  relative  importance  of 
Ei  over  Ej,  so  that  eg  >  1  If  and  only  if  Ei  is  more  important  than  Ej,  it  ia  then  natural  to 
insist  that  the  judge  make  comparison!  in  tuck  a  way  that 

ay  -  —  for  each  ij . 

In  other  words,  such  a  pairwise  comparison  matrix  is  s  judgment  matrix.  The  ideal  pairwise 
comparison  matrix  would  also  be  consistent  For  example,  if  Ei  is  twice  as  important  as  Ej 
and  Ej  ia  three  times  as  important  as  E%,  one  would  expect  Ei  to  be  six  times  as  important  aa 
£*.  However,  human  judgment  is  often  inconsistent,  and  it  is  not  likely  that  a  judge  making 
pairwise  comparisons  will  construct  a  consistent  matrix  except  in  cues  where  the  dimension  is 
small.  A  simple  example  in  which  pairwise  comparisons  do  not  result  in  a  consistent  matrix  is 
that  of  a  tournament:  X  may  win  against  Y  and  Y  against  Z,  but  X  may  lorn  to  Z. 

The  problem  we  consider  ia  this:  Given  an  inconsistent  judgment  matrix  A,  how  can  we 
construct  a  ratio  scale  that  in  some  sense  best  reflects  the  information  in  the  matrix?  Saaty 
(1977b)  proposes  that  the  appropriate  scale  it  the  normelised  eigenvector  corresponding  to  the 
maximal  eigenvalue  of  A. 

Saaty  argues  u  follows:  If  the  judgment  matrix  A  ia  consistent,  then  the  normalised 
eigenvector  corresponding  to  the  single  nonsero  eigenvalue  n  does  give  the  underlying  ratio 
scale.  A  theorem  of  Frobenius  for  matrices  with  positive  entries  (Franklin,  1968)  guarantees 
that  any  judgment  matrix  has  a  positive  eigenvalue  L  that  exceeds  all  the  other  eigenvalues  in 
absolute  value.  This  maximal  eigenvalue  has  an  associated  eigenvector  that  ia  positive  in  all 
its  components. 

Now  an  inconsistent  judgment  matrix  can  be  viewed  u  having  been  derived  from  a  con¬ 
sistent  one  by  perturbation  of  some  or  all  of  the  matrix  component*.  Because  the  eigenvalues 
and  eigenvectors  of  a  matrix  depend  continuously  on  its  components,  small  perturbations  in 
the  components  will  result  in  small  changes  in  tin  eigenvalues  and  eigenvector*.  Thus  when 
the  perturbations  of  the  components  are  small,  the  maximal  eigenvalue  ia  dose  to  ft,  and  the 
corresponding  normalized  eigenvector  is  close  to  the  normalised  eigenvector  of  the  unperturbed 
consistent  matrix.  Therefore,  Saaty  selects  the  suitably  normalized  eigenvector  associated  with 
the  maximal  eigenvalue  as  the  ratio  scale  corresponding  to  the  judgment  matrix. 

Saaty  also  proposes  an  index  of  consistency  for  judgment  matrices.  He  shows  that  an 
nxn  judgment  matrix  whose  only  nonzero  eigenvalue  is  n  must  be  consistent,  and  that  the 
maximal  eigenvalue  L  for  an  inconsistent  judgment  matrix  is  strictly  greater  than  n.  Therefore 
he  uses  the  normalized  difference 


M 


L-n 

n-1 


aa  the  index  of  consistency  of  an  nxn  judgment  matrix  with  maximal  eigenvalue  L.  Notice 
that  tiie  index  is  zero  for  consistent  matrices  and  positive  for  inconsistent  ones.  Saaty  also 
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ihon  that  the  lades  incite eee  at  perturbations  of  the  components  away  from  consistency 
increase. 

Unfortunately,  the  question  of  how  small  the  perturbations  of  matrix  components  must 
be  to  give  rise  to  a  given  deviation  in  the  maximal  eigenvalue  is  a  delicate  one.  Saaty  (1977b) 
describes  an  empirical  investigation  of  this  question  in  which  be  determines  the  consistency 
indices  corresponding  to  randomly  generated  judgment  matrices  of  different  dimensions.  How¬ 
ever,  because  the  eigenvector  does  not  fit  into  any  standard  statistical  framework,  there  is  no 
readily  available  device  against  which  deviations  from  consistency  can  be  measured. 

Saaty  does  show  (1977b)  that  the  consistency  index  n  reflects  the  variance  in  judgmental 
errors  for  an  inconsistent  matrix  in  the  following  sense:  Suppose  that  the  pairwise  compari¬ 
sons  Oy  in  the  judgment  matrix  A  actually  arise  from  perturbations  of  the  ratios  of  components 
of  some  underlying  scale  ,  u*;  i.e., 

Ui 

» 

ui 

1  +  dij. 

Saaty  shows  that  for  small  dg,  2jt  is  an  estimate  of  the  variance  of  the  d#.  Starting  from  this 
estimate,  Saaty  (1977b)  develops  a  test  of  the  hypothesis  of  consistency  for  a  judgment  matrix. 

The  chmce  of  a  scale  to  be  used  in  filling  in  a  pairwise  comparison  matrix  is  somewhat 
arbitrary.  Because  people  find  it  difficult  to  rank  mote  then  about  seven  objects  at  a  time, 
Saaty  recommends  a  subjective  pairwise  comparison  scale  consisting  of  the  integers  from  one 
to  nine  together  with  their  reciprocals.  In  this  scale,  a  value  of  1  is  assigned  to  pairs  of  objects 
that  are  equally  important.  The  integers  8,  5,  7,  and  9  are  associated  with  descriptive  words  (9 
means  “absolute  importance,”  5  means  “essential  or  strong  importance”),  and  the  integers  2, 4, 
6,  and  8  are  used  for  intermediate  values.  Reciprocals  of  integers  are  used  so  that  the  matrix 
of  pairwise  comparisons  is  a  judgment  matrix — that  is,  is  reciprocal  symmetric. 


■  “nr* 


V.  APPLICATIONS  OF  THE  RATIO  SCALE 


Saaty  presents  numerous  applications  requiring  the  estimation  of  ratio  scales  from  pair¬ 
wise  comparison  information  (see  the  bibliography).  He  cites  examples  in  economics,  political 
science,  and  transportation  planning,  as  well  as  in  personal  pUwniwg  areas  such  as  choosing  a 
school  or  a  vacation  spot 

One  of  the  most  interesting  and  useful  applications  of  the  ratio  scale  is  in  the  study  of 
hierarchical  systems.  A  hierarchy  is  a  collection  of  objects  grouped  according  to  levels.  Objects 
at  a  given  level  of  the  hierarchy  depend  on  objects  at  lower  levels.  The  objects  at  one  level 
may  be  ranked  on  a  ratio  scale  according  to  their  importance  relative  to  a  given  object  at  the 
next  higher  level.  Thus  one  may  construct  a  system  of  ratio  scales,  one  scale  for  each  level 
relative  to  every  object  in  the  next  level  up. 

Once  such  a  system  of  ratio  scales  has  been  constructed,  it  can  be  ueed  to  study  interac¬ 
tions  among  all  levels  of  the  hierarchy.  For  example,  in  a  hierarchy  consisting  of  three  levels, 
we  may  determine  the  ranked  importance  of  objects  on  the  lowest  level  relative  to  each  object 
in  the  highest  level.  Suppoee  for  simplicity  that  the  highest  level  counts  of  a  single  object, 
that  the  second  level  has  n  objects,  and  the  third  has  m  objects.  Let  (wi,wt, ...» w.)  be  the 
ratio  scale  that  reflects  the  importance  of  objects  in  the  second  level  relative  to  tin  tingle 
object  in  the  first  level.  Now  the  objects  in  the  third  (lowest)  level  may  be  ranked  on  a  ratio 
scale  relative  to  each  object  on  the  second  level.  Let 

•  •  • » Mmjt  j  ■  1*2, . . . ,  n 

be  the  ratio  scale  for  the  third  level  relative  to  the  /th  object  of  the  second  level  Then,  accord¬ 
ing  to  Saaty,  the  importance  of  objects  in  the  lowest  level  relative  to  the  highest  level  may  be 
measured  by  the  vector  Vi,Vt,  •  •  •  *  vm>  where 

vi  -  »  -1A- 

;-i 

In  matrix  notation,  if  w  -  . . .  ,wH  is  the  ratio  scale  for  the  second  level  relative  to  the 

single  object  in  the  first  level,  and  if 

Mil  **12  ...  uln 

1*21  1*22  ...  **2» 
e  e 

U  - 

e  e 

e  e 

**ml  Wm2  ...  Umn 

is  the  matrix  whose  /th  column  is  the  ratio  scale  for  the  objects  in  the  third  level  relative  to  the 
/th  object  in  the  second  level,  then 


v  -  Uw 


gives  a  scale  of  importance  of  objects  in  the  third  level  relative  to  the  first. 


The  same  procedure  may  be  extended  to  hierarchies  with  more  than  three  levels.  Thus 
knowing  only  the  measures  of  importance  of  objects  in  each  level  relative  to  individual  objects 
in  the  adjacent  higher  level,  we  may  deduce  their  ranked  importance  relative  to  objects  at  all 
higher  levels.  In  particular,  objects  at  the  lowest  level  can  be  ranked  according  to  their  impor¬ 
tance  relative  to  the  object  (or  objects)  at  the  highest  level. 

As  an  example,  consider  the  problem  of  selecting  an  automobile.  (This  example  is  treated 
In  mors  detail  in  Section  DC.)  The  problem  can  be  viewed  in  terms  of  the -hierarchical  struc¬ 
ture  shown  in  Fig.  1.  The  highest  level  of  the  hierarchy  is  the  final  selection  of  the  automobile. 
On  the  second  level  are  attributes  of  the  automobiles— namely  status,  cost,  economy,  and  size. 
The  third  level  consists  of  the  automobile  makes  to  be  considered.  The  automobiles  them¬ 
selves  are  ranked  according  to  each  of  the  attributes,  and  the  attributes  are  ranked  according 
to  their  importance  relative  to  the  overall  objective  of  selecting  a  car. 


Fig.  1— Hierarchy  for  selecting  an  automobile 


Table  I  gives  the  ratio  scales  determined  from  judgments  made  by  one  prospective  buyer. 
The  order  of  automobile  preference  for  this  buyer  is  then  given  by  the  product  of  the  matrix 
and  the  vector  in  Table  1  as  follows: 
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The  final  ranking  reflects  the  buyer’s  perception  of  the  relative  status,  cost,  economy,  and  size 
of  tha  five  automobiles  considered  as  weU  as  his  judgment  of  the  relative  importance  of  these 
few  attributes  in  the  selection  of  an  automobile.  This  buyer’s  first  choice  should  be  T  and  his 
last  choke  C. 


Table  1 


RATIO  SCALES 


Ratio  Seals  of  Second  Level  Relativs  to 

First  Laval 

Status 
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Ratio  Scales  of  Third  Level  Relative  to 
Attributes  at  the  Second  Level 
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VI.  THE  GEOMETRIC  MEAN  SCALE 


Saaty’s  examples  clearly  show  that  the  study  of  interactions  among  various  levels  of  a 
hierarchy  depends  heavily  upon  an  assessment  of  the  ranked  importance  of  objects  at  each 
level  relative  to  objects  in  the  level  above.  The  basic  building  blocks  in  a  hierarchical  study  are 
the  ratio  scales  measuring  the  relative  importance  of  objects  at  a  given  level.  One  would  like 
assurance  that  the  estimates  of  the  ratio  scales  are  well  grounded  in  statistical  theory  and  that 
they  work  well  empirically.  Below  we  propose  a  method  for  constructing  a  ratio  scale  based  on 
pairwise  comparisons.  That  method  seems  superior  to  the  eigenvector  procedure  when  judged 
by  the  criteria  mentioned  above. 

For  n xn  judgment  matrices  A  -  [a#]  and  C  -  [cy],  define 


[n  "11/2 

2  ”  fin  c«) 

It  is  not  difficult  to  verify  that  m  satisfies  the  triangle  inequality  and  is  a  metric  on  the  space 
of  n  xn  judgment  matrices.  Theorem  3  will  show  that  for  any  n  xn  judgment  matrix  A,  there 
is  a  consistent  matrix  C  that  is  m -closest  to  A.  Such  a  consistent  matrix  is  given  by 


where 


n 


/-X 


that  is,  Vi  is  the  geometric  mean  of  the  elements  of  the  ith  row  of  A.  We  will  use  the  vector  v, 
suitably  normalized,  as  the  estimate  of  the  ratio  scale  corresponding  to  A. 

The  following  two  invariance  properties  show  that  m  is  a  suitable  choice  of  metric  for  the 
space  of  judgment  matrices.  Their  proofs  follow  from  the  definition  of  m. 


Theorem  1  (Invariance  under  Transpose),  (i)  Let  A  -  [ay]  and  C  -  [c,y]  be  n  xn  judg¬ 
ment  matrices.  Then  AT  and  CT  are  also  judgment  matrices,  and 

m{AT,CT)  -  m(A,C)  . 


(ii)  Let  A  -  [ay]  be  an  nxn  judgment  matrix,  and  suppose  that  C  -  [c„]  is  the  consistent 
matrix  that  is  m-closest  to  A.  Then  Cr  is  the  consistent  matrix  that  is  m -closest  to  AT. 


Theorem  2  (Invariance  under  Change  of  Scale),  (i)  Let  A  -  [tiy]  and  C  -  [cy]  be  nxn 
judgment  matrices  and  (wi,w2, ...  ,wn)  a  ratio  scale.  Define 

A  -  [oijWi/wj] 

C  -  [caWi/Wj]  . 


Then  A,  C'  are  judgment  matrices,  and 
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m(A,  C)  -  m(A,C)  . 

(ii)  Let  A ,  C,  A',  C'  be  as  in  (i),  and  suppose  that  C  is  the  m-closest  consistent  matrix  to  A. 
Then  C'  is  the  m-closest  consistent  matrix  to  A. 

Recall  that  we  seek  a  procedure  for  associating  ratio  scales  to  judgment  matrices  in  such  a 
way  that  the  ratio  scales  capture  the  subjective  information  inherent  in  the  corresponding 
matrices.  Let  A  be  an  n  xn  judgment  matrix.  Let  C  -  [c»y ]  be  a  consistent  matrix  that  is  m- 
closest  to  A,  and  suppose  that  v  -  vi,v2, . . . ,  u„  is  a  ratio  scale  for  C;  i.e.. 


We  choose  vi,v2, ...  ,un  as  the  estimator  of  the  ratio  scale  corresponding  to  A. 

Under  this  association,  Theorem  1  guarantees  that  the  scale  l/vi,l/v2, . . . ,  l/v„  is  the 
estimator  of  the  scale  corresponding  to  AT.  The  appeal  of  this  invariance  arises  in  a  natural 
way:  Suppose  a  respondent  put  his  estimate  of  Uj/u;-  in  the  position  r#  instead  of  ry.  (There 
appears  to  be  nothing  intrinsically  right  or  wrong  about  recording  the  estimates  this  way.)  In 
that  case,  the  estimation  procedure  should  return  estimates  1/u,-  instead  of  u,-.  With  this  con¬ 
vention  the  estimated  value  of  u,-  should  not  depend  on  an  artifact  of  the  way  the  data  are 
recorded.  It  follows  from  Theorem  1  that  the  GM  procedure  has  this  invariance  property.  The 
EV  procedure  does  not.1 

Additionally,  Theorem  2  guarantees  that  our  choice  of  ratio  scale  is  invariant  under  a 
scale  change  in  the  judgment  matrix.  The  eigenvector  scale  also  fails  this  invariance  test. 

Theorem  3  guarantees  that  the  geometric  mean  scale  gives  the  m-closest  consistent  matrix 
to  any  judgment  matrix. 


Theorem  3.  Let  A  - 
matrix  given  by 


be  an  n xn  judgment  matrix.  Let  C  -  [cy]  be  the  consistent 


Vi 

-  —  » 


where  u;  is  the  geometric  mean  of  the  elements  of  the  tth  row  of  A;  i.e., 

n 

Vi  -  II4/n.  *  -  1,2 ,...,n  . 
i- 1 

Then  m(A,0  is  the  minimal  m-distance  from  A  to  any  nxn  consistent  matrix. 
Proof.  For  any  consistent  matrix  C  -  [cy],  we  can  write 


where  w  -  w\,w2, . . .  ,w„  is  a  ratio  scale.  Thus  we  seek  a  scale  that  minimises  the  expres¬ 
sion 

2  2I£n  °v  _  Wi  ~  tn  “’;)]*  • 

i-l  i>i 

'See  Johneon,  1979,  for  «a  excellent  dUcuaafon  of  this  lack  of  eymmetry  and  it*  rotation  to  the  choice  between  the 
right  and  left  dominant  eigenvector. 


Because  the  estimating  scale  need  be  known  only  up  to  a  scale  factor,  we  may  normalise  by 
imposing  the  aide  condition 

n*®/  “  1 

i-l 

Let 

Yij  -  in  ay,  ij  -  1,2, ....  n  , 
bi  -  fin  u>i,  i  -  1,2, . . . ,  n  . 

Then  the  problem  is  to  minimize 

S  £l>v  -  -  »/>)’ 

i-l  ;> 1 

under  the  side  condition 

-  o  • 

i-l 

Because 

yji  -  -y#  *  J  - 1.2, ....  n 

and 

ya  -  0  , 

this  is  equi/alent  to  minimizing 

5  -  £  £[y«  -  (*,  -  Ml* 

i-l  j-1 

under  the  side  condition 

2*i  -  0  . 

i-l 

Now  5  is  strictly  convex  in  the  differences  6;  -  6;,  and  therefore  strictly  convex  in  the  vector 
6,  so  it  has  a  unique  minimum  at  the  point  where 

-  0  for  i  -  1,2, . . . ,  n  . 

obi 


Setting  these  partial  derivatives  equal  to  zero,  for  k  -  1,2, ....  n 

TK  -  -*2<*  -*  +  *>) 
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and  therefore,  since 


±bj  -  0 


Thus  S  is  minimised  by 


2**  -  • 
j-i 


S>« 

j-i 


i.e.. 


2£n  °« 


fin  «;*  - 


■  ^...,n 


Consequently  the  m-distance  from  A  to  C  is  minimized  by  the  vector  v  given  fay 

^  -  n«r  • 

;'-i 

This  completes  the  proof  of  Theorem  3. 

Recall  that  if  the  matrix  d  is  consistent,  then  the  normalized  geometric  mean  scale 
equal  to  the  normalized  eigenvector  scale.  The  two  scales  are  always  the  same,  regardless 
consistency,  if  the  dimension  is  less  than  or  equal  to  three.  To  see  this  in  the  case  »  -  3,  let 

(1  a 
l/o  1  c 
1/b  1/c  \t 

Then  the  geometric  mean  vector  for  A, 

(  (ab)^\ 
v  -  [  (c/a)1/a  1 
\iubc)l/ y 

is  an  eigenvector  for  A  corresponding  to  the  eigenvalue 


Since  I>  is  of  the  form 


L  -  1  + 


(trw 


i + *  +  — , 

X 

its  value  is  no  less  than  3,  the  dimension  of  A  (with  equality  in  the  consistent  case).  Therefore 
to  is  an  eigenvector  corresponding  to  the  maximal  eigenvalue  for  A.  Hence  in  the  case  n  -  3, 
the  normalized  geometric  mean  vector  and  the  normalized  eigenvector  are  the  same.  This 
result  does  not  hold  for  inconsistent  matrices  with  dimension  greater  than  3. 


8,  sr 


vn.  THE  GEOMETRIC  MEAN  VECTOR  AND  THE 
MAXIMUM  LIKELIHOOD  ESTIMATOR 


We  have  shown  that  given  an  arbitrary  judgment  matrix  A,  the  geometric  mean  vector 
gives  rise  to  the  m -closest  consistent  matrix  to  A.  The  problem  of  representing  a  judgment 
matrix  by  a  ratio  scale  can  also  be  cast  in  the  framework  of  the  general  linear  statistical  regres¬ 
sion  model.  We  will  show  that  under  suitable  assumptions  on  rite  distribution  of  errors  in  the 
expert’s  judgment,  the  geometric  mean  vector  is  the  maximum  likelihood  estimator  for  the 
ratio  scale  corresponding  to  the  judgment  matrix. 

Let  A  -  fa]  be  an  n  xn  judgment  matrix.  We  assume  that  there  is  an  underlying  scale 
(01.102, . . . ,  i0n  whose  ratios  are  perturbed  (by  errors  of  judgment)  to  give  the  elements  of  A, 
namely 


Wi 

fly  ■  —  €(j  9 

Wj  v 

and  thus 

tn  ay  -  in  u>i  -  in  Wj  +  tn  ejj  ,  (7.1) 

i  -  1,2, . . .  ,n;j  >  i  . 

Regarding  the  distribution  of  the  error  term,  the  context  of  the  Saaty-Vargas  (1963) 
approach  assumes  a  multiplicative  model— if  u  -  ultut, ....  u»  is  a  scale  for  the  entities 
{Ei},  then  the  value  of  £?;  relative  to  Ej  is-given  by  Uf/Uf.  Accordingly,  we  have  assumed  that 
errors  are  multiplicative.  Further,  this  context  assumes  that  if  estimates*  tii/u?,  then  1/a $  is 
an  equally  good  estimate  of  uy/u, ;  hence  it  is  appropriate  that  the  distribution  of  ey  be  recipro¬ 
cal  symmetric  in  the  sense  that 


P(a  <  ejj  <:  b)  -  Pla  <  — sfc 
\  ev 


Just  as  the  normal  distribution  is  a  common  model  for  additive  errors,  the  log  normal  dis¬ 
tribution,  for  similar  reasons,  is  a  common  mathematical  model  for  multiplicative  errors.  Addi¬ 
tionally,  it  is  reciprocal  symmetric.  We  assume  that  model  here.  In  Section  VIII  we  evaluate 
the  performance  of  the  GM  estimate  under  a  very  different  error  distribution. 

We  assume  that  the  errors  Cy  are  independent  and  lognormally  distributed  with  means  0 
and  variances  a2.  The  substitution 


«n  au 
Sn  a  i,2 


in 


' 

"£neu 

in  w  i 

in  ei,2 

• 

• 

,  E  - 

• 

• 

in  wn 

• 

in 

gives  the  general  linear  equation 


19 


Y-XB  +  B, 

where  the  matrix  X  has  components  -1, 0,  +1  determined  by  Eq.  (7.1).  In  this  framework  it 
is  well  known  (Scheffe,  1969)  that  the  maximum  likelihood  estimate  for  B  -  [£n  i»jj  is  the 
least-squares  estimate  given  by 


&  -  7T  2*"  * 


and  that  the  estimate  has  all  of  the  usual  desirable  properties  of  least-squares  estimates  under 
the  general  linear  hypothesis  (unbiased,  minimum  variance,  etc.).  Taking  exponentials,  we  find 
that  the  maximum  likelihood  estimate  of  Wj  is  given  by: 


A 

Wi  -  exp 


<£)  -  ik 

i- 1 


(The  same  estimate  is  derived  above  from  the  metric  m  on  the  space  of  judgment  matrices.) 

The  procedure  outlined  above  can  be  modified  to  solve  more  general  estimation  problems. 
For  example,  suppose  that  instead  of  a  single  comparison  for  each  pair  of  objects  Ej  and  Ej, 
there  are  n#  comparisons,  Oy*,  k  -  1, . . . ,  tty  where  may  be  zero  (reflecting  missing  data) 
or  greater  than  one  (reflecting  multiple  comparisons,  say  by  different  judges).  The  problem  is 
then  to  find  a  vector  w  that  minimises  the  sum  of  squares: 

"  9 

S  -  2  2  2)1*"  °ijk  -  (in  Wi  -  in  w;)]  . 
i-i  i> l  *-i 

This  generalization  does  not  yield  a  simple  closed-form  solution  such  as  the  geometric  mean 
vector,  but  in  practice  S  can  be  minimised  and  w  determined  using  standard  least-squares 
regression  packages. 

Regardless  of  the  values  of  rty,  the  geometric  mean  estimation  procedure  leads  to  a 
natural  measure  of  consistency  for  judgment  matrices  that  is  well  grounded  in  statistical  theory 
and  can  be  used  in  hypothesis  testing.  Let  s2  be  the  residual  mean  square 


where  d.f.  is  the  number  of  independent  observations  minus  the  number  of  linearly  indepen¬ 
dent  parameters. 

Note  that  if  rty  -  1,  then 

if.  -  -(»-!)-  -  »<2  -?>  . 

Then  a2  is  an  unbiased  estimator  of  a2  (the  variance  of  the  perturbations)  and  hence  is  a 
natural  measure  of  consistency  of  A. 

Recall  that  if  ny  ■  1,  then  S  can  be  viewed  as  the  squared  distance  from  A  to  the  m- 
cloeest  consistent  matrix.  Therefore  S  is  zero  when  A  is  consistent,  is  close  to  zero  when  A  is 
close  to  consistent,  and  is  far  from  zero  when  A  is  far  from  consistent  Moreover,  because  S 
depends  entirely  on  ratios,  it  is  invariant  under  scale  changes  and  transposes  in  the  sense  of 
Theorems  1  and  2. 


Vm.  EMPIRICAL  COMPARISON  OF  THE  TWO  METHODS: 
A  MONTE  CARLO  STUDY 


The  geometric  mean  vector  gives  an  estimate  for  ratio  scales  based  on  pairwise  compari¬ 
sons  that  is  easy  to  calculate,  satisfies  the  theoretical  requirements  of  invariance  under  scale 
change  and  transpose,  and  is  well  grounded  in  statistical  theory.  It  follows  from  Section  VI 
that  under  certain  assumptions  on  the  distribution  of  the  perturbations,  the  logarithms  of  the 
geometric  mean  vector  components  are  the  minimum  variance  unbiased  estimators  of  the  loga¬ 
rithms  of  the  underlying  ratio  scale  factors.  Thus,  if  errors  have  a  lognormal  distribution,  the 
GM  procedure  provides  the  best  estimate  of  the  underlying  scale  and  in  particular  is  preferable 
to  the  EV  procedure  with  respect  to  the  metric  given  above. 

A  good  procedure  should  be  robust  in  the  sense  that  it  yields  good  performance  when 
measured  by  other  relevant  metrics  and  different  choices  for  the  distribution  of  errors. 
Although  the  robustness  of  the  General  Linear  Model  has  been  widely  treated  in  the  literature, 
we  look  here  at  several  very  specific  comparisons  of  the  performance  of  the  GM  and  EV  pro¬ 
cedures. 

In  this  section  we  give  the  results  of  Monte  Carlo  trials  that  compare  the  EV  and  the  GM 
procedures  under  different  choices  of  error  distributions  and  different  choices  of  metrics. 
These  studies  imply  that  the  EV  is  a  better  estimator  (in  the  sense  of  metrics  considered  here) 
under  both  choices  of  distributions  of  errors.  The  metrics  used  include  measures  of  rank 
preservation,  where  the  EV  has  been  purported  to  be  better  than  the  GM  procedure  (see- 
appendix). 


Throughout  this  section  the  14  will  be  equal  to  i  x  R ,  where  the  constant  R  is  taken  so  that 
the  Ui  sum  to  one. 


Let  {ejj,  i,j  -  1, . . . ,  n,  j  >  i}  be  a  collection  of  positive  independent  random  variables 
drawn  from  a  suitable  population;  we  construct  a  perturbed  matrix  D  with  elements 


~ea  for  i 


1  for  i 


1,2, . . . ,  n,  j  >  i 

1,2, . . . ,  n,  j  <  1 

1,2, . . . ,  n,  j  -  i 


D  is  a  judgment  matrix,  but  because  of  the  perturbations  it  is  typically  not  consistent.  Any 
estimate  of  an  underlying  ratio  scale  for  D  should  give  rise  to  a  consistent  matrix  that  is  in 
some  sense  close  to  A.  Two  quite  different  metrics  for  closeness1  are  used  in  the  Monte  Carlo 


*It  hat  bam  mentioned  in  the  literature  (Saaty  and  Vargas,  1984,  and  others),  that  Fichtnar  (1984)  haa  developed  a 
metric  m  with  the  proparty  that  the  m -closest  consistent  matrix  to  a  subjective  judgment  matrix  is  the  consistent 
matrix  corresponding  to  the  maximal  eigenvector.  Typically  a  metric  is  of  interest  because  it  provides  an  intuitive 
understanding  of  the  topology  of  a  space— La.,  it  tails  you  what  points  are  close  to  (neighborhoods  of)  other  points, 
hence  what  sequences  converge,  and  to  what  points.  Unfortunatsty,  Fichtner's  complicated  metric  yields  the  dice  rets 
topology,  a  very  pathological  example  wherein  every  point  is  an  open  eat,  the  only  sequences  that  converge  an  thoaa 
whose  terms  an  all  the  tame  from  some  point  on,  and  even  Saaty  and  Vargas’s  (1984)  normalised  limit  of  An  doss  not 
exist  It  is  the  tarns  topology  given  by  the  metric  that  call*  the  distance  from  two  points  1  if  the  points  are  not  equal 
and  0  if  they  an 


a 


study  described  hwr  the  sum  of  squares  of  errors  (SSE) 


SSE  -  j\(h-Ui)  , 

i-l 


and  the  sum  of  squares  of  the  non  of  logarithms; 


SSEL 


n 

-2 


(in  Ui-  in  Uj)  , 


when 


Uj, »'  -  1,3, ...,•» 

is  the  actual  nonaaBaed  ratio  scale,  and 

^  0  "  1,2,  ••*,/! 

is  the  estimated  ratio  scale.  In  addition  we  have  looked  at  several  measures  of  order  preserva¬ 
tion,  where  the  EV  procedure  is  said  to  be  preferable  to  the  OM  procedure. 

The  choice  of  population  from  which  the  multiplicatively  perturbing  random  variables  e# 
are  drawn  should  reflect  that  in  a  judgment  matrix  the  distribution  of  errors  should  be  recipro¬ 
cal  symmetric  as  mentioned  above: 


P(a  <  eg  x  fr)  •  P^«  <  i  s  b  j  . 

We  have  used  two  convenient  and  very  (Hsaimilar  distributions  satisfying  this  property:  a  log¬ 
normal  distribution  whose  underlying  normal  distribution  has  mean  aero,  and  a  distribution 
obtained  from  the  ratio  of  two  independent,  uniformly  distributed  (on  fad)  where  0  <  e  <  d) 
random  variables,  We  consider  perturbations  drawn  from  populations  with  distributions  of 
both  of  those  types. 

We  computed  the  sums  of  squares  of  errors  and  sums  of  squares  of  errors  of  logarithms 
for  individual  trials  and  then  totaled  them  over  the  trials  for  both  the  geometric  mean  vector 
and  the  eigenvector  estimates. 

In  a  comparieon  of  two  estimation  techniques  A  and  B,  it  it  common  to  look  aft  the  aver¬ 
age  or  sum  of  performance  measures  over  some  large  number  of  trials  or,  whan  poasibia,  the 
axpectad  performance  with  respect  to  some  distribution  of  errors.  Because  most  researchers 
ate  not  faced  with  the  problem  of  routinely  analysing  thousands  of  subjective  judgment 
matrices,  these  results  may  be  misleading*  Although  A  may  be  preferable  to  B  in  the  sense  of 
expected  errors,  B  may  bt  preferable  to  A  90  percent  of  the  time,  the  eaaming  contradiction 
bring  explained  by  B  occasionally  giving  large  outliers.  When  an  outlier  is  suspected,  the 
researcher  may  reject  that  method  on  that  trial  and  look  at  other  methods.  Because  of  these 
concerns  we  have  soored  each  of  the  Monte  Carlo  trials  and  computed  the  frequency  with 
which  the  geometric  mean  outperforms  the  eigenvector.  The  perverse  situation  mentioned 
above  does  not  seem  to  occur  here.  Within  experimental  error,  the  geometric  mean  performs 
_  as  well  as,  or  better  than,  the  eigenvector  in  the  sum  of  errors  and  outperforms  it  in  60  percent 

|\  or  more  of  the  trials. 


* 


St 


Compaurieoa  with  Susa  of  Squared  Log  Error  and  Lognormal  Errors 

Table  2  gives  the  results  of  the  trial  that  will  theoretically  favor  the  geometric  mean  vec¬ 
tor.  lognormal  errors  and  the  metric  given  by  the  sum  of  the  squared  errors  of  logarithms. 
The  entries  in  the  first  column  describe  the  parameters  of  the  Monte  Carlo  run.  The  first 
number  is  the  dimension  of  the  scale,  the  second  number  is  the  variance  of  the  logarithm  of 
the  error  term  The  next  term  is  the  average  “consistency  ratio”  for  the  trial— that  is,  the 
difference  between  the  maximal  eigenvalue  and  the  dimension,  normalised  by  the  dimension 
minus  1.  We  are  uncertain  about  what  the  consistency  ratio  really  means,  but  the  eigenvector 
is  said  to  do  well  when  the  consistency  ratio  is  less  than  .10.  Accordingly  we  have  concen¬ 
trated  on  runs  in  this  realm.  The  next  number  gives  the  number  of  trials.  In  the  cases  with 
very  small  log  perturbation  error  we  used  6000  trials,  in  the  cases  where  the  trend  seemed  clear 
we  ueed  1000  trials.2 

The  eigenvector  outperforms  the  geometric  mean  in  every  case,  in  terms  of  both  sum  of 
errors  and  the  percentage  of  trials  where  it  was  closer  to  the  underlying  consistent  matrix. 

Table  2 

RATIO  SCALES  PERTURBED  BY  LOGNORMAL  ERRORS,  SUM  OF  SQUARED 
ERRORS  OF  LOGARITHMS,  MONTE  CARLO  COMPARISON  OF  GEOMETRIC 
MEAN  VECTOR  AND  EIGENVECTOR 


Dimension, 

Variance, 

(L  -  n)/(n  -  l), 
Number  of  Trials 

Sum  of  Squared 
Error  of  Lots, 
Geometric  Maun 

Sum  of  Squared 
Error  of  Lop, 
Kfeaiwartor 

Peroontupo  of 
Thais  in  Which 
Geometric  Mean  Is 
Better  in  8quarad 
Error  of  Loft 

5/.01/.008/9000 

43.4 

424 

61 

5/.04/.012/6000 

170.8 

171.7 

62 

6/.06/.028/6000 

377.6 

382.1 

63 

5/.16/.061/1000 

136.2 

138.4 

67 

8/.28/.060/1000 

310.4 

2184 

60 

6/.49/.166/1000 

427.0 

448.6 

66 

6/1.0/ .349/1000 

8634 

047.7 

61 

7/.01/.006/8000 

463 

464 

63 

7/D4/.016/6000 

1783 

180.6 

66 

7/.09/.03S/6000 

4034 

4104 

66 

7/.16/.069/1000 

1604 

1644 

66 

7/46/.006A0Q0 

228.7 

236.1 

60 

7/.49/J08/1000 

440.6 

4864 

61 

7/1.0/.446/1000 

036.2 

1102 

67 

10/.01/.040/6000 

46.4 

464 

63 

10/.04/ .016/5000 

186.4 

187.6 

66 

10/.09/.OS6/6000 

4214 

488.4 

60 

10/.16/C67A000 

147.7 

1664 

68 

10/46/.106/1000 

2344 

261.6 

66 

10/.40/423A000 

4664 

6344 

60 

10AO/416AOOO 

046.1 

1266 

78 

*n»M 
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The  preference  for  the  eigenvector  is  weakest  when  the  dimension  or  the  error  is  small,  as 
would  be  expected  because  the  methods  give  the  same  answer  when  the  dimension  is  3  or  less, 
or  when  the  matrix  is  consistent 

Comparison  with  Sum  of  Squared  Log  Error  and  Batio  of  Uniform  Errors 

Table  3  presents  the  same  information  for  the  case  of  perturbations  by  ratios  of  uniform 
random  variables.  The  variance  given  is  (as  above)  the  variance  of  the  logarithm  of  the  pertur¬ 
bation  term.  The  consistency  ratio  is  determined  empirically  and  may  differ  in  the  tl 
decimal  place.  Even  with  these  very  different  error  terms,  the  geometric  mean  vector  is  pie 
able  in  every  case  and  in  both  measures.  Again,  the  degree  of  preference  is  small  when 
methods  can  be  expected  to  give  answers  that  are  very  close  and  becomes  much  stronger  as 
dimension  or  the  variance  of  errors  grows. 


Tables 

RATIO  SCALES  PERTURBED  BY  RATIOS  OP  UNIFORM  RANDOM  VARIABLES, 
SUM  OP  SQUARED  ERRORS  OP  LOGARITHMS,  MONTE  CARLO  COMPARISON 
OF  GEOMETRIC  MEAN  VECTOR  AND  EIGENVECTOR 


Dimenrion, 

Variance, 

(L  -  n)/(n  -  1), 
Number  of  Trial* 

Sum  of  Squared 
Error  of  Loe*. 
Geometric  Mean 

Sum  of  Squared 
Error  of  Log*, 
Eigenvector 

Percentage  of 
Trial*  in  Which 
Geometric  ||mb  1b 
Better  in  flqmnd 

Error  of  Log* 

5/.01/.003/5000 

42.3 

42.3 

60 

5/.04/.012/5000 

1674 

168.1 

61 

5/.09/.027/5000 

386.6 

386.4 

60 

5/.16/.060/1000 

134.4 

136.0 

61 

5/.25/.081/1000 

2144 

2194 

66 

5/. 49/, 161/1000 

426.1 

442.2 

66 

6/1.0/460/1000 

867.4 

969.7 

66 

7/.01/.004/6000 

444 

444 

61 

7/.04/.014/6000 

177.6 

1774 

62 

7/.09/.033/6000 

406.2 

4104 

63 

7/.16/ .069/1000 

146.0 

146.0 

61 

7/.25/.094/1000 

2264 

2294 

66 

7/.4S/. 196/1000 

463.3 

492.6 

66 

7/1.0/. 461/1000 

906.1 

1119 

72 

10/41/.004/5000 

46.2 

464 

62 

10/D4/.016/6000 

186.6 

186.4 

63 

10/.09/.087/6000 

4164 

420.7 

66 

10/.16/ .066/1000 

1484 

1614 

66 

10/.28/.106/1000 

230.3 

239.1 

61 

10/.49/419/1000 

460.2 

606.9 

66 

10/1.0/419/1000 

9634 

1274 

32 

mi 
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Table  4  and  5  consider  the  sum  of  wpiwd  wor  aw trie  instead  of  the  wa  iqmwd  error 
of  logarithms.  Lognormal  (non  havt  baan  used  in  Table  4  and  tha  ratio  of  uniform  etna  in 
Tabla  6.  In  this  matric,  which  diflam  aubatantialiy  from  tha  metric  uaad  to  justify  tha 
gernetric  mean,  tha  geometric  moon  stifl  outperforms  tha  aifanvactor  in  both  measure  in  both 
table  in  every  eae  except  om,  In  tha  “arrant"  cam  tha  aigsnvactor  doe  batter  by  “1”  in  tha 
iaaat  significant  digit.  (Hen  tha  tend  of  tha  resuHa,  wa  conclude  that  this  aberration  is  tha 
rasuit  of  aaperimantal  arror.  Tha  patten  mentioned  above  repeats  itself:  The  big  difference 
occur  when  the  dimension  at  the  variance  of  tha  errors  become  large. 


Table  4 


RATIO  SCALES  PERTURBED  BY  LOGNORMAL  ERRORS,  SUM  OP  SQUARED 
ERRORS,  MONTI  CARLO  COMPARISON  OF  GEOMETRIC  MEAN  VECTOR 
AND  EIGENVECTOR 


1 

- 1). 

Sum  of 

Squamd  Error, 
GmmMc  Mean 

Sum  of 

Squared  Error, 
Eis*n vector 

Pwewueof 

Trials  is 

Which  Gaomatric 

Mae  is  least 

fti 

l 

5/.01/.003/5000 

1.70 

1.70 

51 

s. 

5/.04/.012/8000 

ast 

6M 

52 

i 

b/mfjm/Yxn 

16.0 

15.1 

63 

1 

5/.1S/C61/1000 

6.1* 

3.16 

fit 

1 

>/m/eo/iooo 

a.16 

MO 

56 

s/.ea/.iae/ioeo 

14.1 

16J 

54 

6/1.0/JH9/1000 

8U 

34J 

58 

7/.01/.006/6000 

1.00 

1.01 

62 

7/.04/.015/B000 

am 

AM 

53 

7/.00/.OS3/5000 

AM 

0.11 

64 

7/.16/.069/1000 

Ml 

3.43 

66 

7/.25/.096/1000 

4.61 

5.08 

67 

7/.49/.203/1000 

0.77 

10.7 

68 

7/1.0/.445/1000 

10.4 

23.3 

61 

10/.01/.040/6000 

.638 

.639 

62 

i 

10/.04/.016/6000 

2.15 

2.18 

63 

9 

10/.08/.036/6000 

4.90 

5.04 

56 

10/.16/.067/1000 

1.73 

IBS 

59 

H 

10/.26/.108/1000 

2.71 

2.90 

61 

1 

10/.49/.223/1000 

5.16 

5.88 

64 

9 

10/1.0/^16/1000 

10.5 

14.1 

71 

25 


Table  5 

RATIO  SCALES  PERTURBED  BY  RATIOS  OF  UNIFORM  RANDOM  VARIABLES. 
SUM  OF  SQUARED  ERROR.  MONTE  CARLO  COMPARISON  OF  GEOMETRIC 
MEAN  VECTOR  AND  EIGENVECTOR 


Dimension, 

Variance, 

(L  -  n)/(n  -  1), 
Number  of  Triala 

Sum  of 

Squared  Error, 
Geometric  Mean 

Sum  of 

Squared  Error, 
Eigenvector 

Percentage  of 
Trials  in 

Which  Geometric 
Mean  is  Least 
Squares  Better 

5/.OI/.003/5000 

1.71 

1.71 

50 

5/.04/.012/5000 

6.73 

6.75 

61 

5/.00/027/5000 

15.7 

15.6 

50 

5/. 16/. 060/1000 

5.36 

5.39 

51 

5/.26/.081/1000 

8.40 

&5S 

53 

5/.49/.161/1000 

16.1 

16.6 

56 

5/1.0/.360/1000 

31.2 

36.1 

62 

7/.01/.004/5000 

.996 

.996 

60 

7/.04/.014/6000 

3.96 

3.96 

50 

7/.09/.033/6000 

9.07 

9.13 

53 

7/.16/.069/1000 

3.13 

3.14 

51 

7/.25/.094/1000 

4.92 

4.98 

52 

7/.49/.196/1000 

9.80 

10.6 

61 

7/1.0/.461/1000 

19.1 

24.0 

70 

10/.01/.004/5000 

.531 

.531 

50 

10/.04/.016/6000 

2.14 

2.16 

51 

10/.09/.037/5000 

4.76 

4.79 

53 

10/.16/.066/1000 

1.73 

1.75 

52 

10/.25/.106/1000 

2.67 

2.75 

56 

10/.49/.219/1000 

5.29 

5.71 

61 

10/1.0/.519/1000 

10.7 

14.4 

74 

IX.  EXAMPLE 


In  this  section  we  discuss  the  use  of  the  eigenvector  and  the  geometric  mean  vector  in  a 
specific  subjective  judgment  situation.  Estimates  of  underlying  utility  vectors  and  consistency 
values  derived  from  the  two  methods  are  compared. 

Consider  the  automobile  selection  problem  introduced  in  Sections  II  and  V.  The 
hierarchical  structure  for  this  problem  was  given  above  in  Fig.  1. 

Subjective  judgment  data  for  this  example  were  obtained  from  one  prospective  buyer. 
The  buyer  made  pairwise  comparisons  reflecting  his  perceptions  of  the  relative  importance  of 
the  attributes  of  status,  cost,  economy,  and  size  in  selecting  an  automobile.  Judgments  were 
made  based  on  the  subjective  judgment  scale  developed  by  Saaty  (1977b).  The  buyer  made 
comparisons  in  such  a  way  that  the  resulting  pairwise  comparison  matrix  would  be  reciprocal 
symmetric  (i.e.,  a  judgment  matrix).  The  resulting  judgment  matrix  A  is: 


Status 

Cost 

Economy 

Size 

Status 

1 

1/6 

1/5 

1/2 

Cost 

5 

1 

1 

1/3 

Economy 

5 

1 

1 

1/2 

Size 

2 

3 

2 

1 

Next,  we  constructed  judgment  matrices  from  the  buyer’s  pairwise  comparisons  of  the  five 
types  of  automobiles  relative  to  each  of  the  two  attributes  of  status  and  size: 

Subjective  Comparison  Relative  to  Status 


H 

T 

M 

D 

C 

H 

1 

2 

1/9 

1/8 

1/7 

T 

1/2 

1 

1/9 

1/8 

1/7 

M 

9 

9 

1 

6 

4 

D 

6 

6 

1/6 

1 

1/7 

C 

7 

7 

1/4 

7 

1 

Subjective  Comparison  Relative  to  Size 


H 

T 

M 

D 

C 

H 

1 

1/2 

7 

3 

8 

T 

2 

1 

8 

4 

9 

M 

1/7 

1/8 

1 

1/3 

3 

D 

1/3 

1/4 

3 

1 

4 

C 

1/8 

1/9 

1/3 

1/4 

1 

t 


We  calculated  the  eigenvector  and  the  geometric  mean  vector  separately  for  each  of  the 
three  subjective  judgment  matrices  above.  Resulting  scale  estimates  for  the  two  techniques  are 
given  in  Table  6. 
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Notice  that  the  scales  determined  by  the  two  methods  are  very  close  in  value.  This  is  as 
expected  from  theoretical  considerations,  because  the  two  methods  give  the  same  results  for 
consistent  matrices  and  should  agree  closely  for  nearly  consistent  ones. 

The  results  in  Table  6  indicate  that  the  prospective  buyer  considers  automobile  size  to  be 
considerably  more  important  than  status,  cost,  or  economy.  He  ranks  T  as  having  the  best  size 
of  any  of  the  cars  under  consideration. 

Because  exact  values  were  available  for  cost  and  economy  of  the  five  automobile  types,  it 
was  not  necessary  to  compute  scales  for  them  from  pairwise  comparisons.  The  normalised 
scale  values  for  the  automobiles  relative  to  these  two  attributes  were  determined  to  be: 


Cost  Ratings 

Economy  Ratings 

H 

.2703 

.3357 

T 

.4196 

.2929 

M 

.0554 

.1071 

D 

.1571 

.1500 

C 

.0976 

.1143 

Table  6 

COMPARISON  OF  SCALE  ESTIMATES  FOR  THE  AUTOMOBILE  EXAMPLE 


Normalised 


Matrix 

Geometric 

Mean 

Consistency 

s2 

Normalized 

Eigenvector 

Consistency 

a 

Matrix  A 

Statue 

I  1/5  1/5 

1/2 

.0612 

.5680 

.0686 

.1513 

Coat 

1  1 

1/3 

.2453 

.2426 

Economy 

1 

1/2 

.2716 

.2579 

Size 

1 

.4019 

.4112 

Matrix  B 

Statue 

H 

1  2  1/9  1/6 

1/7 

.0409 

.5550 

.0378 

.1770 

T 

1  1/9  1/6 

1/7 

.0310 

.0294 

M 

1  6 

4 

.5307 

.5239 

D 

1 

1/7 

.1132 

.1131 

C 

1 

.2842 

.2958 

Matrix  C 

Size 

H 

11/27  3 

9 

.3162 

.1150 

.3143 

.0417 

T 

18  4 

9 

.4632 

.4630 

M 

1  1/3 

3 

.0681 

.0696 

D 

1 

4 

.1299 

.1288 

C 

1 

.0336 

.0343 

.0343 
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From  each  method,  we  now  have  scale  estimates  for  the  five  automobile  types  relative  to 
each  of  the  four  automobile  attributes,  as  well  as  a  scale  of  importance  of  the  four  attributes. 
Scale  estimates  for  the  five  cars  relative  to  each  attribute  are  used  as  columns  of  a  5  x  4 
matrix.  This  matrix  is  multiplied  by  the  4-dimensional  vector  of  importance  of  attributes. 
The  resulting  5-dimensional  vector  reflects  the  prospective  buyer’s  ranking  of  the  five  automo¬ 
biles.  The  calculations  for  both  methods  are  carried  out  in  Table  7. 


Table  7 

DETERMINATION  OF  RANKINGS  OF  FIVE  AUTOMOBILES 


& 

& 


Maks  Criterion  Final 

of  Car  Status  Coat  Economy  Site  Scale  Scale 


Appendix 


COMPARISON  OF  RANK  PRESERVATION  ABILITIES 


We  believe  the  issue  of  rank  preservation  is  a  red  herring:  The  EV  procedure  has  been 
recommended  for  analyzing  hierarchical  structures  and  quantifying  judgments.  A  procedure 
designed  to  do  well  in  a  cardinal  estimation  problem  ought  not  be  judged  on  the  basis  of  its 
capabilities  in  ordinal  or  rank  estimation  problems.  If  the  researcher  has  an  ordinal  estimation 
problem  it  would  be  better  and  easier  to  ask  the  respondent  to  rank  the  entities  in  order  of 
preference. 

Further,  the  oft-mentioned  ability  of  the  EV  to  do  well  in  this  context  seems  to  be  based 
(Saaty  and  Vargas,  1984)  on  the  following  argument: 

There  is  a  natural  way  to  derive  the  rank  order  of  a  set  of  alternatives  from  a  pairwise  com¬ 
parison  matrix  A.  The  rank  order  of  each  alternative  is  the  relative  proportion  of  its  domi¬ 
nance  over  the  other  alternatives.  This  is  obtained  by  adding  the  elements  in  each  row  in  A 

and  dividing  by  the  toted  over  all  rows.  (Emphasis  added.) 

Everything  that  has  been  written  in  the  present  context  about  the  analysis  of  subjective 
judgment  matrices  has  dealt  with  multiplicative  problems.  The  estimates  are  of  ratios,  not  of 
sums  or  differences.  In  this  context  determining  rank  order  by  adding  elements  seems  unnatu¬ 
ral.  The  natural  ranking  inherent  in  a  subjective  judgment  matrix  for  a  multiplicative  problem 
is  that  obtained  by  the  product  of  the  row  entries,  not  the  sum.  The  GM  procedure  gives  pre¬ 
cisely  that  ranking,  and  it  outperforms  the  EV  procedure  in  this  measure  also. 

Although  we  believed  the  issue  was  not  worth  treating,  referees  objected  to  the  omission 
of  rank  preservation  properties  in  the  comparisons  of  the  two  methods,  hence  its  inclusion 
here.  In  Tables  A.1  and  A.2  we  have  measured  the  ability  of  the  GM  and  EV  procedures  to 
preserve  rank.  The  scenarios  are  as  above:  Table  A.l  uses  lognormal  errors  and  Table  A.2 
ratios  of  uniform  random  variables.  Recall  that  the  underlying  scales  iq  are  proportional  to  i 
and  scaled  so  that  they  sum  to  one. 

Rank  is  typically  expressed  as  a  vector;  we  know  of  no  single  overwhelmingly  natural 
scalar  measure  of  rank  error.  We  have  used  two  very  different  measures,  both  simple,  but 
lacking  simple  descriptions.  To  fix  ideas,  let  us  assume  that  the  dimension  of  the  scale  is  5. 
We  represent  the  ranking  in  the  underlying  scale  by  1,2, 3, 4, 5.  If  we  estimate  this  scale  by  a 
vector  v  that  satisfies 

Vi<V2<V6<V3<  l>4  » 

then  we  represent  the  ranking  of  v  by  1,2, 5, 3, 4.  If  v  preserves  the  ranking  inherent  in  u,  its 
ranking  will  be  the  same  as  that  of  u. 

We  have  called  the  measures  of  rank  preservation  used  below  the  sum  of  rank  reversals 
and  the  sum  of  weighted  rank  reversals.  The  sum  of  rank  reversals  is  just  the  number  of  times 
we  count  a  reversal  in  ordering  as  we  go  from  left  to  right  ("1”  in  the  above  example,  because 
5  >  3).  The  weighted  rank  reversal  measures  the  degree  of  displacement  from  the  rank  of  the 
underlying  scale.  In  the  above  example  it  is  given  by 

(1  -  l)a  +  (2  -  2)a  +  (3  -  5)a  +  (4  -  3)a  +  (5  -  4)a  -  6. 
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Table  A.1 


RATIO  SCALES  PERTURBED  BY  LOGNORMAL  ERRORS,  RANK  PRESERVATION 
ERROR,  MONTE  CARLO  COMPARISON  OF  GEOMETRIC  MEAN  VECTOR 
AND  EIGENVECTOR 


Dimension, 

Sum  of 

Sum  of 

Sum  of  Weighted 

Variance, 

Sum  of  Rank 

Rank  Reversal 

Weighted  Rank 

Rank  Reversal 

(L  -  n)/(n  -  1), 

Reversal  Errors, 

Errors, 

Reversal  Errors, 

Errors, 

Number  of  Trials 

Geometric  Mean 

Eigenvector 

Geometric  Mean 

Eigenvector 

6/.01/.003/8000 

2 

2 

4 

4 

6/.04/.012/5000 

268 

260 

616 

520 

5/.09/.028/5000 

963 

971 

1964 

1980 

5/.16/.051/1000 

210 

209 

426 

422 

6/.25/.080/1000 

529 

538 

1160 

1190 

5/.49/.166/1000 

825 

847 

2002 

2060 

6/1.0/.349/1000 

1081 

1141 

2874 

3036 

7/.01/.006/5000 

14 

14 

28 

28 

7/.04/.015/5000 

683 

694 

1372 

1394 

7/.09/.033/5000 

2084 

2143 

4384 

4510 

7/.16/.059/1000 

712 

739 

1588 

1650 

7/.26/.095/1000 

911 

962 

2178 

2298 

7/.49/.203/1000 

1332 

1376 

3548 

OCQi 

7/1.0/.446/1000 

1775 

1826 

5322 

6642 

10/.01/.040/5000 

74 

77 

148 

154 

10/.04/.016/5000 

1809 

1824 

3684 

3730 

10/.09/.036/5000 

4350 

4446 

9590 

9832 

10/.16/.067/1000 

1317 

1367 

3174 

3330 

10/.26/.108/1000 

1659 

1710 

4334 

4534 

10/.49/.223/1000 

2189 

2282 

6460 

6948 

10/1.0/.516/1000 

2667 

2852 

9228 

10266 

In  Tables  A.1  and  A.2  we  have  given  the  sum  of  rank  reversal  errors  and  the  sum  of 
weighted  rank  reversal  errors.  (In  the  tables  in  this  report  we  started  each  new  Monte  Carlo 
process  with  a  new  random  number  seed  to  avoid  repeatedly  using  a  sequence  of  random 
numbers  that  may  favor  one  procedure  or  the  other.) 

In  Table  A.1,  with  lognormal  perturbations,  in  only  two  out  of  42  comparisons  does  the 
eigenvector  outperform  the  geometric  mean.  In  four  other  comparisons  the  methods  tied.  In 
the  remaining  36  pairs,  the  GM  procedure  outperforms  the  EV  procedure,  and  the  differences 
in  performance  become  more  pronounced  as  the  dimension  and  underlying  variance  increase. 

In  Table  A.2,  where  the  perturbations  are  ratios  of  uniform  random  variables,  the  eigen¬ 
vector  also  does  poorly,  although  a  little  better  than  above.  Again,  the  relative  performance  of 
the  geometric  mean  improves  as  the  dimension  and  the  variance  of  the  errors  increase.  Even 
in  these  trials,  which  are  dominated  by  the  low  variance  and  consistency  ratios  that  are  said  to 
favor  the  eigenvector,  the  few  occasions  where  the  eigenvector  outperforms  the  geometric  mean 
seem  to  be  the  result  of  random  aberrations,  not  of  any  inherent  qualities  that  are  pertinent  to 
estimating  ratio  scales. 


RATIO  SCALES  PERTURBED  BY  RATIOS  OF  UNIFORM  RANDOM  VARIABLES, 
RANK  PRESERVATION  ERROR,  MONTE  CARLO  COMPARISON  OF 
GEOMETRIC  MEAN  VECTOR  AND  EIGENVECTOR 


Dimension, 

Variance, 

(L  -  n)/(n  -  1), 
Number  of  Trials 

Sum  of  Rank 
Reversal  Errors, 
Geometric  Mean 

Sum  of 
Rank  Reversal 
Errors, 
Eigenvector 

Sum  of 

Weighted  Rank 
Reversal  Errors, 
Geometric  Mean 

Sum  of  Weighted 
Rank  Reversal 
Errors, 
Eigenvector 

5/.01/.003/5000 

0 

0 

0 

0 

6/.04/.012/5000 

262 

248 

604 

496 

5/.09/.027/6000 

1069 

1074 

2164 

2172 

5/.16/.050/1000 

343 

340 

724 

716 

5/.25/.061/1000 

633 

639 

1144 

1178 

5/.49/.161/1000 

779 

824 

1890 

2002 

6/1.0/.360/1000 

1077 

1125 

2888 

3094 

7/.01/.004/5000 

4 

4 

8 

8 

7/.04/.014/5000 

686 

702 

1374 

1410 

7/.09/.033/5000 

2164 

2176 

4630 

4576 

7/.16/.069/1000 

716 

699 

1668 

1548 

7/.26/.094/1000 

963 

938 

2286 

2288 

7/.49/.196/1000 

1322 

1362 

3532 

3706 

7/1.0/.451/1000 

1692 

1794 

6116 

5758 

10/.01/.004/5000 

72 

70 

144 

140 

10/.04/.016/5000 

1760 

1756 

3594 

3598 

10/.09/.037/6000 

4188 

4228 

9220 

9364 

10/.16/.066/1000 

1296 

1284 

3078 

3112 

10/.26/.106/1000 

1632 

1661 

4264 

4400 

10/.49/.219/1000 

2161 

2212 

6434 

6844 

10/1.0/.619/1000 

2714 

2917 

9406 

10802 
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